-
Notifications
You must be signed in to change notification settings - Fork 73
error reduction from short lived process #997
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
error reduction from short lived process #997
Conversation
b8beba1 to
f4844ee
Compare
|
@prashantbytesyntax Please fix the linter issue |
|
@prashantbytesyntax and @ashokbytebytego, many tests still failed. One possible cause is below. Please revise. CRITICAL: gprofiler.profilers.factory: Couldn't create the Perf profiler, not continuing. Run with --no-perf to disable this profiler\nTraceback (most recent call last):\n File "gprofiler/profilers/factory.py", line 54, in get_profilers\nTypeError: SystemProfiler.init() got an unexpected keyword argument 'min_duration'\n' |
|
@prashantbytesyntax @ashokbytebytego, I still see the linter issue with the PR. Please fix it. And please rebase your PR with the latest master which probably can resolve one test failure. |
|
@mlim19 I have ran lint and tests locally and test as well. Please let me know if I miss anything here? source venv311/bin/activate && ./lint.sh --ci. - for lint ource venv311/bin/activate && sudo -E $(which python3) -m pytest -v tests/test_short_lived_process_fix.py warnings.warn(PytestDeprecationWarning(_DEFAULT_FIXTURE_LOOP_SCOPE_UNSET)) tests/test_short_lived_process_fix.py::TestShortLivedProcessFix::test_custom_min_duration_threshold PASSED [ 6%] =============================================================================================================================================================== warnings summary =============================================================================================================================================================== tests/test_short_lived_process_fix.py:22 -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html |
This implementation adds "Smart Skipping Logic" to gProfiler to reduce error rates when profiling short-lived processes, addressing the issues described in Intel gProfiler Issue #996.
Description
Problem Statement
Currently gProfiler has high error rates during profiling due to:
Short-lived processes: Profilers attempting to profile processes that exit during profiling
Impact: Multiple errors/day from rbspy, py-spy failing on transient processes
Root Cause: Race conditions with process lifecycle
Solution: Smart Skipping Logic
Core Implementation
Process Age Checking: Skip processes younger than min_duration seconds
Enhanced Error Handling: Graceful handling for processes that exit during profiling
Applied Across: Ruby, Java, and Python profilers
How Has This Been Tested?
unit tests
x86_64, arm_64
`
Use default 10 second threshold
./gprofiler
Custom threshold - skip processes younger than 5 seconds
./gprofiler --min-duration 5
More aggressive - skip processes younger than 30 seconds
./gprofiler --min-duration 30
`
Screenshots
Checklist: